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INTRODUCTION 



Two of the roost dramatic recent innovations in educational technology 
are computer-assisted instruction and simulation games. In CAI, the student 
learns by responding to stimuli presented under the control of the computer 
program in such a way that the student's response determines the next 
stimulus to be presented. In a simulation game, the student learns by 
adopting a role in a simulated environment and making decisions which, along 
with chance and the decisions of other players, determine the outcome of the 
game. 

Computer games combine these essential characteristics of the two tech- 
niques. They differ from other simulation games largely in the complexities 
of the situations and the amount of arithmetic computations they permit. 

Like other forms of CAI and unlike most other games, they are often designed 
to be played by one student at a time. However, unlike the most coomon form 
of CAI - programned instruction - the student's responses are not definitely 
right or wrong (although, in any game situation, some decisions may be clearly 
better than others). 

Like other games, computer games offer the player a choice of strategies, 
and the best strategy is not obvious. This feature of computer games should 
(we reasoned) provide an advantage to the student who has the opportunity 
to discuss his responses with a teammate before typing them into the computer. 
Although the most enthusiastic supporters of CAI stress the advantages of one- 
to-one interaction between student and computer, we felt that in a computer 
game, interaction between students would prove more beneficial than one-to- 
one student-computer interaction. (There is, of course, an obvious economic 
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advantage in having students play a computer game in groups, rather than 
individually, since the cost of computer time can be divided by the number 
of students using the machine at a time.) 

Our hypotheses were explicitly stated as follows: 

1. Students who play a computer game in teams of two or three will 
perform at least as well as those who play the game individually, on a 
test designed to measure learning from the game. 

2. Teams of two or three students will be at least as successful in 
the game as individual students. 

The reader will note that, in order for the results of the study to 
support both of these hypotheses, performance in the game will have to be 
positively correlated with performance on the test. Logically, one would 
expect students who learned most from the game to be able to use their 
knowledge to improve their performance in the game. However, two charac- 
teristics of educational games may counteract this effect. The first is 
that a player who employs risky strategies in a game, especially when 
playing it for the first time, is likely to fail spectacularly; yet, be- 
cause of his more varied experiences, he is likely to learn more from the 
game than a player who uses more conservative strategies. The second is 
that players can learn to use strategies within the context of a game and 
still be unable to explain their strategies in words. ^ Thus, it would 
not be paradoxical for the results of the study to support only one of the 
hypotheses stated above. 

Can learning activities designed for individual students working 
alone be made more effective by administering them to small groups? We 
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found a number of studies which seemed relevant to this problem (although 
most of them came to our attention only after we had conducted our ex- 
periment). Dick (1963) and Sawaris (1966) found differences favoring 
group presentation; Klausmeier jet al. (1963) and Noble (1967) found 
differences favoring individual presentation; and Hudgins (1960), Grubb 
(1964), King (1967), and Napier (1967) found no significant differences 
favoring either mode. The subjects for these studies ranged from fifth 
grade through college, and the learning tasks varied greatly in length, 
subject matter, and means of presentation. However, the results did not 
vary consistently with any of these factors. 
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METHODOLOGY 



The subjects were 44 sixth-grade studdnts from a school in the 
Baltimore area. They were members of two special mathematics classes 
for high-ability students: a class of 22 boys and a class of 22 girls. 

The mean IQ (Kuhlmann-Anderson) for all 44 students was 132. (S.D. = 

13.3) In addition to IQ scores, the students' scores on four measures 
of academic ability and achievement were obtained. /.II of the students 
were familiar with computer games, having played them at least twice 
before at the Center, but none had ever played this particular game. 

The game used for this study was the Surfboard Game, developed by 
J inner M. Leonard, of BOCES (Board of Co-operative Educational Services, 
xorktown Heights, New York). In this game, the player takes the role of 
the manager of a company which manufactures styrofoam surfboards. He is 
required to decide how many workers he will employ and how much he will 
charge for his product. If his accumulated profits increase to a certain 
level, he is also given the option of buying new machinery. The routines 
in the game which provide for unplanned expenses occurring at random were 
removed for this study, since they would have introduced an uncontrolled 
variable. The criterion used to measure performance in the game was the 
average monthly increase (or decrease) in the player's net assets. 

The test used to measure learning from the game consisted of three 
parts. The first part contained six multiple-choice items designed to 
measure the student's understanding of certain economic relationships 
which function in the game. These questions were phrased so as to re- 
quire the students to make predictions similar to those that would be 



necessary for rational planning in the game. The second part consisted 
of three items designed to measure the student's grasp of certain accounting 
concepts which he should have used in making his decisions in the game. 

The third part was an open-ended question which asked the student to ex- 

O 

plain his strategy for setting prices in the game. 

The students were segregated by sex in their regular classes and were 
kept that way for the study. Within sex groups, they were assigned ran- 
domly to four treatment groups. The students in Group 3 played the game 
in teams of three, those in Group 2 played in teams of two and those in 
Group 1 played alone. All the students in these three groups took the 
test immediately after playing the game. The students in Group 0 did not 
play the game but took the test, with the open-ended question phrased as 
a hypothetical situation. Groups 1, 2 and 3 played the game and took the 
test at the Center; Group 0 took the test at school. The students were 
told not to discuss the game with other students, and we could find no 
evidence to indicate that they did discuss it. 

Six students (all of the same sex) played the game at a time: one 
team of three, one team of two and one single student. The instructions 
for the game were printed and positioned at the computer terminals.^ No 
additional instruction or help was given to the students. The students 
played the game for an hour and were allowed enough time to answer all 
the items on the test. 

Of the three computer terminals available for this study, two were 
teletype machines connected to a time- sharing system and one was a con- 




5 



sole typewriter connected to the Center's IBM 1401. In order to avoid 
introducing a consistent bias into the study because of any difference 
in equipment, the machines were assigned as in Table l„ 
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RESULTS 



Table 2 shows the mean scores, by sex and treatment group, for Parts 
I and II of the test. Two-way analysis of variance showed no significant 
main or interaction effects for sex or treatment group on either part of 
the test. Even the difference on Part I between Group 0 and 1 is not 
significant.^ 

Table 3 shows the mean scores, by sex and treatment group, foi. each 
of five antecedent measures. The verbal and quantitative scores from the 
Kuhlmann-Anderson Test have been converted from percentiles to T-scores to 
permit statistical manipulation. The achievement test scores for paragraph 
meaning and applied arithmetic are expressed as they were reported to us, 
in grade-level equivalents. 

Table 4 shows the zero-order correlations, for all subjects taken to- 
gether, between Parts I and II of the test and the five antecedent measures. 
On the basis of these correlations, we selected paragraph-meaning achievement 
and quantitative ability as control variables for an analysis of covariance, 

by treatment group, of the scores for Part I. The analysis of covariance 

^ 6 

did not yield a statistically significant treatment effect. 

A content analysis of the students' responses to the open-ended question 
(Item 10) produced the results shown in Table 5. There is no reason that a 
student could not have mentioned all three concepts; yet not one subject 
mentioned both seasonal demand and cost of production. Of course, it was 
possible to answer the question without mentioning any of the three concepts 
listed, and several subjects did so. 

Table 6 contains a summary of the students ' performance in the game 
itself, with each team (of 1, 2, or 3 students) counted as a single player. 
"Game score" is defined as the average montnly increase in the player's net 
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assets. "Price range" and "employment range" are two independent measures 
of a pi oyer's willingness to experiment with extreme strategies. "Price 

» 

range" is simply the difference between the highest and lowest prices 
* charged by the player, and "employment range" is the difference between 

the highest and lc^st numbers of ma^ line operators he en^>loyed. "Rate of 
play" is counted in game months per hour of actual play.^ 

The results shown in Table 6 were tested for significance by means 
of a two-way analysis of variance for each of the four variables. Only 
two effects were significant. The tendency of boys to play the game faster 
than girls was significant at the .01 level (F = 10.02, df = 2 and 12). 

Also, the effect of interaction between sex and treatment group on "price 
range" was significant at the .05 level (F - 5.40, df = 2 and 12). However, 
this second result is difficult to interpret and may well not hold true for 
larger samples. 

9 

Intercorrelations were computed for the four game variables and Parts I 
„ and II of the test (using the mean score for each team for these last two 

variables). The results are shown in Table 7. The only correlation that 
approaches significance is that between "price range" and "employment range". 
A high positive correlation here could be expected, since each of these 
variables is an indication of the same general tendency on the part of the 
player . 

Finally, a comparison of the students using the 1401 with those using 
the teletype is shown in Table 8. No significant difference between students 
using the different machines was found on either part of the test or on any 
of the four game variables. 
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DISCUSSION 



The purpose of the study was to detect the educational advantages 
of group play (as contrasted with individual play) in a computer game. 

If these advantages exist, our study failed to find them. 

There are two ways of measuring learning in a game situation. One 
is to observe the students' performance in the game; the other is to 
test them outside the game for the knowledge or skills they were to have 
acquired by playing the game. Our study showed no significant differences 
on either criterion between students who played in groups and those who 
played alone. What differences there were tended to favor the groups of 
three when game performance was the criterion and the individual players 
when test performance was the criterion. However, these differences were 
neither consistent nor statistically significant. 

One result of this experiment may provide some insight into the kind 
of learning that takes place in a computer game. The differences in the 
scores for Part I of the test between Groups 1, 2, and 3, which played the 
game, and Group 0, which did not, are almost entirely accounted for by 
Item 2 of the test. This item requires the student to predict the effect 
on total raw material costs of a change in the number of production workers. 
The relationship is somewhat indirect; hiring more production workers en- 
ables a company to increase production, which requires more raw materials, 
which increases raw material costs. While the students who did not play 
the game had to try to "reason out" this relationship, those who played 
the game experienced it directly. The result was that none of the students 
who did not play the game answered this item correctly, while every student 
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who played the game alone answered the item correctly. (Before conducting 
the experiment, we asked five or six adults to take the test. The only 
one who answered this item correctly was an experienced accountant.) 

The boys in our sample played the game significantly faster than the 
girls. (The means were 17.83 game-months per hour for the boys and 14.25 
for the girls.) However, we do not feel justified in generalizing this 
finding to other situations, since our subjects were all of high intelligence 
and all familiar with computer games (though not with this one). And of 
course, even among these same subjects the girls might have played faster 
than the boys if we had chosen a different computer game. 
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SUMMARY 

Sixth-graders of high academic ability were divided into four groups. 
Three of the groups played a computer game: one group played alone, one 
played in pairs, and one in threes. The fourth group did not play the 
game. All the subjects then took a test designed to measure learning 
from the game. No significant differences in learning were observed, but 
there was a statistically significant tendency of boys to play the game 
faster than girls. A difference in machines used to play the game pro- 
duced no consistent or significant effects. 



TABLE 1 



ASSIGNMENT OF SUBJECTS TO DIFFERENT MACHINES 



Day Qroup 1 Group 2 Group 3 





1 


1401 


TTY 


TTY 


Boys 


2 


TTY 


1401 


TTY 




3 


TTY 


TTY 


1401 




4 


1401 


TTY 


TTY 


Girls 


5 


TTY 


1401 


TTY 




6 


TTY 


TTY 


1401 



TABLE 2 



TEST PERFORMANCE - MEANS 









n 


Part I 


Part II 


Group 


0 


Male 


4 


3.25 


4.22 






Female 


4 


3.25 


4.67 


Group 


1 


Male 


3 


4.17 


4.33 






Female 


3 


4.33 


4.00 


Group 


2 


Male 


6 


3.67 


4.50 






Female 


6 


3.00 


2.50 


Group 


3 


Male 


9 


3.89 


6.25 






Female 


9 


3.44 


3.50 



Total Group 0 


8 


3.25 


4.88 


Total Group 1 


6 


4.50 


4.17 


Total Group 2 


12 


3.33 


3.50 


Total Group 3 


18 


3.67 


4.44 


All Male Subjects 


22 


3.77 


4.68 


All Female Subjects 


22 


3.45 


3.77 


Total All Subjects 


44 


3.61 


4.23 



TABLE 3 



ANTECEDENT MEASURES - JEANS 







n 


V 


- Q 


IQ 


PM 


AA 


Group 0 


Male 


4 


60.3 


61.0 


137 


11.6 


11.9 




Female 


4 


54.6 


55.5 


129 


10.1 


9.8 


Group 1 


Male 


3 


50.7 


60.2 


132 


11.4 


11.3 




Female 


3 


61.6 


52.4 


130 


9.9 


8.8 


Group 2 


Male 


6 


55.8 


57.4 


134 


10.6 


11.4 




Female 


6 


56.9 


54.1 


130 


10.6 


9.8 


Group 3 


Male 


9 


52.4 


59.6 


131 


11.2 


11.0 




Female 


9 


58.4 


54.0 


131 


11.0 


9.6 


Total Group 


0 


8 


57.5 


58.2 


133 


10.8 


10.8 


Total Group 


1 


6 


56.1 


56.3 


131 


10.6 


10.1 


Total Group 


2 


12 


56.3 


55.7 


132 


10.6 


10.6 


Total Group 


3 


18 


55.4 


56.8 


131 


11.1 


10.3 


All Male Subjects 


22 


54.5 


59.3 


133 


11.1 


11.3 


All Female Subjects 


22 


57.8 


54.1 


130 


10.6 


9.6 


Total All Subjects 


44 


56.2 


56.7 


132 


10.9 


10.4 



V Verbal Ability (Kuhlmann - Anderson Test) 

Q ~ Quantitative Ability (Kuhlmann - Anderson Test) 
IQ = IQ Score (Kuhlmann - Anderson Test) 

PM = Paragraph Meaning 
AA = Applied Arithmetic 
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TABLE 4 



INTERCORRELATION OF TEST PORTIONS 
AND ANTECEDENT VARIABLES ( n = 44) 





I II V 


Q 


IQ 


PM 


AA 


I 


1.00 .34 .34 


.27 


.35 


.42 


.26 


II 


1.00 .36 


.51 


.51 


.46 


.40 


V 


1.00 


.38 


.78 


.40 


.35 


Q 




1.00 


.82 


.33 


.63 


IQ 






1.00 


.45 


.59 


PM 








1.00 


.46 


AA 










- -1.00 


V 


= Verbal Ability (Kuhlmann - 


Anderson Test) 






Q 


Quantitative Ability (Kuhlmann - Anderson Test) 






IQ 


IQ Score (Kuhlmann - Anderson Test) 








PM 


= Paragraph Meaning 










AA 


= Applied Arithmetic 











TABLE 5 



STUDENTS ' RESPONSES TO TEST ITEM 10 

PROPORTION MENTIONING RELEVANT 
ECONOMIC CONCEPTS 



Group 





0 


1 


2 


3 


Seasonal Demand 


.000 


.167 


.417 


.222 


Cost of Production 


.625 


.000 


. 417 


.278 


Price - Quantity 
Relationship 


.375 


.500 


.750 


.389 



Note: The concept of seasonal demand was not as relevant to the 

question answered by Group 0 as it was to the question 
answered by the groups that played the game. 



TABLE 6 



Group 1 Male 

Female 

Group 2 Male 

Female 

Group 3 Male 

Female 

Total Group 1 
Total Group 2 
Total Group 3 

All Male Subjects 
All Female Subjects 
Total All Subjects 



GAME PERFORMANCE - MEANS 



n 


Game 

Score 


Price 

Range 


3 


2215 


3.53 


3 


1581 


2.17 


3 


1294 


2.93 


3 


2351 


1.20 


3 


4013 


1.11 


3 


1214 


6.25 


6 


1898 


2.85 


6 


1822 


2.05 


6 


2613 


3.68 


9 


2507 


2.52 


9 


1715 


3.21 


18 


2111 


2.86 



Employment 

Range 


Rate of 
Play 


6.67 

9.00 


22.33 

13.67 


4.67 

2.33 


15.67 

13.67 


7.33 

16.33 


16.00 

11.33 


7.83 


18.00 


3.50 


14.67 


11.83 


13.67 


6.22 


18.00 


9.22 


12.89 


7.72 


15.44 
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TABLE 7 



INTERCORRELATIONS OF TEST PORTIONS 
AND GAME VARIABLES (n = 18) 





I 


II 


Test, Part I 


1.00 


.44 


Test, Part II 




1.00 


Game Score 







Price Range 
Employment Range 
Rate of Play 



GS 


PR 


ER 


R/P 


-.12 


.03 


-.18 


.02 


.42 


.29 


.16 


.20 


1.00 


.16 


.59 


.26 




1.00 


.67 


.42 






1.00 


.15 








1.00 



TABLE 8 
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FO OTNOTES 



For a more thorough explanation of these characteristics of 
games and of the ways in which games function as learning 
devices, see Coleman, 1967 and 1968, and McKenney and Dill, 
1966. 

Appendix A contains a sample printout for this game. 

Appendix B contains a sample copy of this test, with the 
correct answers to items 1-9 indicated. 

These instructions were as follows: 

HELLO: 

IN THE GAME YOU ARE ABOUT TO PLAY, YOU WILL ASSUME THE 
ROLE OF THE OWNER OF A FACTORY THAT MANUFACTURES STYROFOAM 
SURFBOARDS. YOU HAVE 30 WORKERS, 10 OF THEM ARE SHIPPERS, 

10 OF THEM ARE PACKERS AND 10 OF THEM ARE OPERATORS. 

YOU HAVE 5 MACHINES PRODUCING SURFBOARDS, AND BY HIRING 
OR FIRING OPERATORS YOU CAN VARY PRODUCTION AND PRODUCTION 
COSTS. SINCE YOU HAD TO BORROW MONEY FROM THE BANK TO BUY 
THESE MACHINES, YOU HAVE TO PAY BACK THE BANK $10,000 EVERY 
DECEMBER. 

YOU CAN MAKE DECISIONS, SUCH AS WHETHER YOU WILL HIRE 
OR FIRE ANYONE, AND HOW MUCH YOU WILL CHARGE FOR THE SURF- 
BOARDS. WHEN THE COMPUTER WANTS YOU TO MAKE A DECISION, IT 
TYPES A ? LIKE THIS: ? 

THE SUMMt'.v SEASON LASTS FROM MARCH THROUGH SEPTEMBER. 

NOW GO OUT AND MAKE A MILLION, AND GOOD LUCK. 

By the use of Scheffe's method for post hoc comparisons, a 
difference of 1.89 would be required for significance at the 
.05 level. 

The internal consistency of Part I of the test, as measured 
by Kuder-Richardson Formula 20, is .276. This value trans- 
lates by means of the Spearman- Brown Formula, into an esti- 
mated average item- intercorrelation of .057. Thus, these 
six items are almost unrelated. 

F = 1.83; an F-ratio of 2.85 is required for significance at 
the .05 level. The adjusted means for Groups 0,1,2, and 3 
were 3.23, 4.59, 3.45, and 3.57. 
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APPENDIX A 



Sample monthly report from the game 



BEGINNING INVENTORY, ( 516 ) + MONTHS PRODUCTION ( 1650 ) = 
2166 SURBOARDS TO SELL THIS MONTH. 

THIS MONTH'S SALES 2061 BOARDS AT $9.85 EACH 
TOTAL RECEIPTS $ 20300.9 

INVENTORY REMAINING AT END OF MONTH = 105 

RAW MATERIALS 1650 BOARDS AT $4 EACH = $ 6600 
LABOR COSTS 33 WORKERS AT $200 EACH = $ 6600 
FIXED EXPENSES $2275 

TOTAL MONTH'S EXPENSES $ 15475 



PROFIT (SALES - EXPENSES) 

LAST MONTH :$ 5565 
THIS MONTH :$ 4825.85 

CHECKING ACCOUNT BALANCE : 

LAST MONTH :$ 10565 
THIS MONTH :$ 15390.8 

LAST MONTH 10 OPERATORS WITH 5 MACHINES PRODUCED 1500 BOARDS 
THIS MONTH 13 OPERATORS WITH 5 MACHINES PRODUCED 1650 BOARDS 

DO YOU WISH TO HIRE OR FIRE ANY MACHINE OPERATORS? 

TO HIRE, TYPE IN + AND THE NUMBER. 

TO FIRE, TYPE IN - AND THE NUMBER. 

TO LEAVE THE SAME, TYPE IN 0. 

?0 



LAST MONTH YOU SOLD 1984 BOARDS AT $ 10 EACH. 

THIS MONTH YOU SOLD 2061 BOARDS AT $ 9.85 EACH. 

HOW MUCH WILL YOU CHARGE FOR EACH SURFBOARD NEXT MONTH? 
?10. 25 



APPENDIX B 



Test used to measure learning from the game. 

Answer all questions even if you are not sure of the answer. 
Hark your answers directly on the question sheet. 



If 3 bicycle manufacturing company hires more production 
workers, what will happen to its total labor cost? 

(wO It will go up. 

( ) It will go down. 

( ) It will stay the same. 

( ) You can't tell. 

2. If the company hires more production workers, what will happen 
to its total raw material cost ? 

(✓) It will go up. 

( ) It will go down. 

( ) It will stay the same. 

( ) You can't tell. 

3. If the company hires more production workers, what will happen 
to its fixed expenses (for example, rent, lighting, property 

taXCS ) ? 

( ) They will go up. 

( ) They will go down. 

( •/) They will stay the same. 

( ) You can't tell. 

4. If the company raises the selling price of its bicycles, which 
of the following things is least likely to happen? 

( S') The company will sell more bicycles. 

( ) The company will sell fewer bicycles . 

( ) The company will sell the same number of bicycles. 

5. Suppose the company is selling all the bicycles it makes, and It 
hires more production workers. Which of the following things is 
least likely zo happen? 

( ) The company will sell more bicycles. 

( *0 The company will sell fewer bicycles. 

( ) The company will sell the same number of bicycles. 

6. Suppose the company is selling all the bicycles it makes. What 
will happen to its profits if it lowers the selling price of its 
bicycles . 

( ) They will go up. 

( ✓) They will go down. 

( ) They will stay the same. 

( ) You can't tell 



7. On the following list, check only those things the company would 
need to know in order to find its total costs of production for 
one month. 

( ) Number of bicycles produced during the month 

( ) Number of bicycles sold during the month 

( ) Selling price of each bicycle 

( ) Inventory on hand at the beginning of the month 

(/) Total labor cost for the month 
(S) Total raw material cost for the mcnth 
( S) Fixed expenses for the month 

8. On the following list, check only those things the company would 
need to know in order to find the cost of producing each bicycle . 

( •'O Number of bicycles produced during the month 

( ) Number of bicycles sold during the month 

( ) Selling price of each bicycle 

( ) Inventory on hand at the beginning of the month 

( S ) Total labor cost for the month 
(S) Total raw material cost for the month 
(s) Fixed expenses for the month 

9. On the following list, check only those things the company would 
need to know in order to find the number of bicycles it had left 
at the end of the month. 

(S) Number of bicycles produced during the month 

(S) Number of bicycles sold during the month 

( ) Selling price of each bicycle 

(S) Inventory on hand at the beginning of the month 
( ) Total labor cost for the month 

( ) Total raw material cost for the month 

( ) Fixed expenses for the month 

(Note: For items 7-9, the student received one point for each choice 
correctly checked, minus one point for each choice incorrectly 
checked. ) 

10. When you played the Surfboard Game, how did you decide how much 
to charge for your surfboards? How would you do it if you had 
another chance to play the game? 

(Subjects who did not play the game answered the following alternate item. ) 

10. How do you think the bicycle manufacturing company ought to decide 
how much to charge for its bicycles? 



